Design and implementation of message-passing services for the Blue Gene/L supercomputer

of message-passing services for the Blue Gene/L supercomputer G. Almási C. Archer J. G. Castaños J. A. Gunnels C. C. Erway P. Heidelberger X. Martorell J. E. Moreira K. Pinnow J. Ratterman B. D. Steinmacher-Burow W. Gropp B. Toonen The Blue Genet/L (BG/L) supercomputer, with 65,536 dualprocessor compute nodes, was designed from the ground up to support efficient execution of massively parallel message-passing programs. Part of this support is an optimized implementation of theMessage Passing Interface (MPI), which leverages the hardware features of BG/L. MPI for BG/L is implemented on top of a more basic message-passing infrastructure called the message layer. This message layer can be used both to implement other higher-level libraries and directly by applications. MPI and the message layer are used in the two BG/L modes of operation: the coprocessor mode and the virtual node mode. Performance measurements show that our message-passing services deliver performance close to the hardware limits of the machine. They also show that dedicating one of the processors of a node to communication functions (coprocessor mode) greatly improves the message-passing bandwidth, whereas running two processes per compute node (virtual node mode) can have a positive impact on application performance.